After I updated my OS from Windows 10 to Windows 11, "open file location" disappeared from search result sometimes.

Sometimes, it shows me "open file location" like this so that I can open the containing folder.

This is how Windows 11 works. Right Click and like operations are context sensitive.

So in your first screen shot, the selection was an App and the contexts provided are sensible for Apps.

In the second screen shot, the selection is a copy function because that is all that is sensible for a document. You already know where the document is located so "Open File Location" does not apply in most operations.

Windows 11 is full of context sensitive selections which vary on what you are trying to do.

What you see above is normal for Native Windows 11 as you have presented it.